June 2002

Last summer at the O'Reilly Open Source Convention, Brian Behlendorf told me he expected the majority of products sold by "open source" companies would soon be closed -- or at least closed to everybody who isn't a customer. It was at that same convention that Microsoft's Craig Mundie defended his company's "shared source" development approach, in a debate with Michael Tiemann of Red Hat. While there were plenty of arguments over broad philosophical differences and licensing particulars, it was hard to argue with certain facts that nobody wanted to talk about. Just about every surviving company in the open source movement was already selling (and nearly all were certainly using) closed source software. And whether we called these new approaches "shared source" or "gated source," it was clear that development models were moving toward middle ground from all directions, including Microsoft's.

But nobody was more willing to talk about it than Brian Behlendorf. As founder of the Apache Web Server Project and CollabNet, Brian has been a leading voice in the open source movement, yet perhaps also the member most interested in exploring the ways in which commercial and open source development both work together and divide labors: one side creating common infrastructure for everybody, the other creating commercial products for customers.

"Apache" was brief for"a patchy server". It succeeded because it was highly practical and adaptive. In that respect, it resembled its progenitor, and his approach to problems. In talking with Brian, I hoped to get a similar tack on how an open source company does business in a new frontier where open source is part of -- but not the entire -- ecosystem.

- Doc Searls

DS: How does what you're doing with source code differ from what VA is doing with SourceForge?

BB: I feel like what we're doing is pretty different. From my perspective open source is more a process than an ideology. It's almost a form of investment for a company. You invest in creating IP and giving it out to the world. You expect it, just like planting seeds, to fertilize somewhere and get more resources, and come back stronger than it was when you put it out there. And if that's not happening you have to reconsider what you're doing.

DS: But the ideology runs strong. One conversation overwhelms the other.

BB: Exactly. I was on a panel at LinuxWorld about the state of open source. I wanted to talk about the kinds of issues we're facing today that are new or that we need to think about as a community, and it seemed like all anybody else wanted to tlak about was the GPL. Business are facing hard real-world choices here about what to sell and what to give away; and they should be able to make different decisions for different pieces of their software stack.

DS: It seems to me where we are right now is: the development bazaar meets the market bazaar, and to survive open source companies are going to have to figure out what works for both.

BB: That's right.

DS: Is what you're doing essentially a new category of open source?

BB: I don't think it's any different from what any standard company that's starting to incorporate open source software is going through. It's the same kind of thing that goes through IBM's mind when they say "What part of WebSphere doe we open source, and what do we keep to ourselves, or plan to open up at some later point. Those are basically the same kind of decisions we're making, with the extra twist that our customers get our source. Maybe there are other people in our community who are not our customers, but are subject to the same conditions. It's been an evolution from the perspective that we can do everything as an ASP, and all the underlying bits would be free, to torglan where you need a combination of open bits and bits that you call your own, wethr you're an ASP or whether you're a licensed option model. Every ASP out there is being pressured to make their software usable "behind the firewall" in a typically hosted environment. If all that underlying software is open source it becomes very difficult to foil? support under contracts for it; it becomes to set up systems integrator relationships because the more you invest in making systems integrators successful the more able they are to be independent from you. And certainly there is a free rider problem still that needs to be addressed. The model where you have all your software free ... even though the moral imperative is there, perhaps, where hey, people should feed back into the troughs that they pull out from... While I'm comfortable trying to ask for that sentiment on things like Apache, when you're talking about enterprise software and software that doesn't have the kind of community that Apache has, or that is much more complex than Apache, the critical mass might never be achieved to the point where systems integrators see it's in their interests to do that. Whereas keeping any modifications themselves or whatever.

DS: What about distribution. There is in the open source definition a clause that says you can't distribute this. If you think you need two-tier distribution, you'll need something that a lot of other people can sell.

BB: What we want is for lots of people to use the software and building on top of it, so they realize the value of giving back. With Apache, lots of ISPs hit bugs and came back with patches. That's a characteristic of software that should be open sourced. It's more infrastructural than end-user oriented. Or the user base is likely to be pretty broad and the users are likely to be pretty technical, and where you've got a justification for the ammount IP you're giving away that says "Hey, by giving this away we've got this other business that will really benefit when there are lots of people using our stuff. So there are all these different levels to it.

As an example, there are two packages that we do want lots of people using that we still have out there in the open source community: Subversion, our ne version-control system and Scarab, our next-generation bug-tracking software (both at Tigris.org). That's because our value is going to be in having really nice tools. But if we're going to still have semi-standard tools we're going to benefit greatly by the fact that CVS is still a pretty important part of our infrastructure, and everybody is comfortable with CVS. Not everybody loves it, or even likes it, but it's out there, tested, and meeting 90% of their need for version control. That seems like a fair statement to make. Most of the companies we go into and try to sell our stuff to are already using CVS. So all those things help us tremendously.

So we want the same thing to happen with Subversion, which we hope will be the next version of CVS, because it solves the same problem, and we already know there are companies out there that want to use it -- like VA. We can justify that effort based on all these other benefits it will have. That's not the case for some other piece of software. And what happens, so to speak, looking at the whole stack of stuff that we'fe releasing, is the question, "Are we really getting the type of community involvement that justifies having the code out there as open source? Is it worth the situations we're running into where the fact that this stuff is open actually hurts us?

DS: Such as?

BB: We have people going, "Well, I've got some guys sitting around. Their project was cancelled because the economy sucks and I can just have them download the software, and go ahead." From our perspective, okay, they should do that, but certainly that means this is a customer we cant' get any revenue from. So it became tough for us to say that all of our stuff had to be out there where anybody could grab it for free. The original strategy had been to put out a framework that had a mixture of components that could plug into that framework, like Subversion, like Scarab, some of which were open, some of which were proprietary, or gated, or whatever you want to call it. And it would be a mixture of all three of those that would be the product that we sold, and it was the the proprietary bits that gave up the secret sauce that made people come to us rather than just downloading our software and becoming a competitor.

DS: That was the original plan.

BB: Yes, but we looked at the resources that we've been able to put toward building this software, that we put into the framework -- and by the framework I mean some infrastructural parts but also parts like the administrative UI and the Web pages that handle users and projects and things like that, tying all these things together -- plus the free components, such as Subversion and Scarab. And we saw that this was simply overwhelming our ability to invest in more proprietary parts. And that was just the reality of the fact that getting the framework right is always more complex than you think it's going to be as an engineer and we really wanted to keep Scarab and Subversion open source. We realized we were 95% to 98% open. And we needed some way to move more towards the middle. We have so much work that we needed to do on the bits that were open that we couldn't really talk about shutting any of those down and stopping work on them. So it came down to a matter of taking some of these things we were working on and moving them into the closed space. Moving Subversion there made no sense because we already had very active outside contributions on it. And that really is a piece we want others out there using. It was really this other piece, Helm, the UI framework, that made the most sense considering making part of our core gated IP, rather than an open source project. That was also the project that had the least outside participation. We have a few comments in the discussion, but that was about it. Helm was also the most complex piece, among others that were not necessarily desinged to work together in the first place. We were also starting on our next generation of stuff. So now was a good time to start reducing our give-away.

DS: How do you maintain the different groupings. What's where?

BB: We've always tried to maintain a kind of separation between church and state. Whether that's a real separation or not you can decide for yourself. My view has always been that independent open source communities need their own identity. I'm not a fan of companies registering .com for the commercial stuff and .org for the commuity, because that looks like all the work is for one company's IP. And one company's brand. So Tigris.org has always been separate. We do have a little Collab.net banner on it, but it presents itself as an independent thing. Which it is. We had thought of some models where Tigris.org for the software would actually be some sort of free for noncommmercial use license sitting on Tigris.org, but we decided instead that moving it off Tigris.org and moving it onto one of our internal development machines -- actually to an extranet that's manned at Collab.net, would be better because we wouldn't corrupt the positioning of Tigris.org as an open source project. And it actually opens up the possibilty of putting some new projects on there that aren't Collab.net projects. Some of our people come from the academic world and have some ideas for opening up projects there for computer science graduate students to do research and give them a home for software they write, and some analysis tools. Make it a home for academic discussion of software development tools. It's independent from Collab.net and remains that way. So we went through a bunch of decision making about how best to do this and what its ramifications would be, and I think we made the right decision along those lines.

DS: I'm trying to get a schematic picture of what's closed, what's open and what's gated here. Is "gated" the right term?

BB: I don't like it because it connotes suburban America, pretending something is out of sight, out of mind.

DS: It's unfortunate that Microsoft kind of nailed the literal meaning with "shared source."

BB: Yeah, shared source is definitely a much better term for it. But I don't know if that's exactly accurate in their case becasue the IP is not shared. The term might be more interesting if there were a system where you could have lots of different IP -- a shared source community.

DS: Do you share the IP with your customers?

BB: We're taking our time talking to customers about that. There are some, for example, that might not be Collabnet customers, but would be able to contribute in other ways. There are complexity issues. How do you assign a dollar amount to a patch? Or even to a new feature? We're still just doing a lot of talking to customers and potential customers.

DS: Do customers actually want IP?

BB: This is one strategic benefit of open source. Our customers have basically been saying that so long as they have an expert agreement where if we go out of business they get some rights to the code in perpetuity. That's good. That's something we'd be pretty comfortable doing. We just haven't worked out other issues such as what happens when a customer writes something and puts it into our gated community. What'll happen to the IP on that? There they might want to start by assigning the IP to Collabnet and perhaps roll out a model after that. Especially since we're working to improve the modularity of our architecture right now. Putting in an extra model for Jabber, for example. But it's not clean yet.At some point in the future there will be a cleaner plug-in infrastructure. When that happens you can see it as a marketplace for components and stuff. So Customer A can conttibute component tht implements some sort of analytics, then Customer B says they want it. We install it for Customer B, manage the infrastructure nd charge them extra and some of the revenue goes to Customer A. That could be interesting. We'll get to that point.

DS So if I'm looking at your list of products, which is the plug-in architecture?

BB: The name of that particular thing is Helm. It used to be on Tigris.org. We also removed a couple other projects that didn't make sense outside the context of Helm. The release engineering project and a sandbox project, both of which were just around getting this stuff built as a suite and managed and stuff like that. Although the fork that we took with Bugzilla, which we renamed Issuezilla, because it's more of an issue-tracking tool built to talk to Helm to get permissions and stuff, didn't mnake sense to sit out there s an open source project. And there is no dependency on Helm for any of the projects at Tigris.org now. What you'll find there are standalone, independent tools. Which is important to us. This was another thing that made me more comfortable with this idea. People thought, "Whoa, do I have to pull down this whole suite to be able to use any of these tools?" The answer then was yes. But it's clearer now that those are independent tools that are independently useful.

DS: Does Helm manifest on the public site at CollabNet, or is it strictly internal?

BB: It's not on the public site. We have an extranet for customers who have a name and a password. And if you're a customer you can see the source code to it. If you're not, you don't. It is no longer part of our mission to go out and get a lot of people using these tools. One of the models we did consider was free redistribution, but you had to have the right to use the license in order to use it legally. And I think there was some concern that if we did that we'd have to help people to install it for nothing in return. Yes, some of those people might be potential customers, but we didn't have the manpower to be able to help a bunch of people install software -- especially softare that admittedly isn't easy to install. We might return to that when we have something that's so easy it you can make, install and away you go. But right now we're just built to work with customers.

DS: If a customer comes along and wants to join your gated community, what are they buying? Is it solutions, products, or what?

BB: The gated community is not a product in itself. We're not trying to build a developer network.

DS: You're trying to get customers and have relationships with them.

BB: Right. And one useful aspect of being a customer of ours is getting access to the source code that we use. Which is interesting, because almost all of our customers are ASPs, which almost never give their source code out anyway. So there is an advantage in that. And we help our customers set boxes on their own infrastructure if they want to have a place that they can kind of stage some ideas. And I'm definitely interested in having all our customers able to understand the code and fix bugs. I want to do the same kind of things that you do in open source projects where somebody might not only give in bug reports but also say 'I think the problem is in this line and here's a patch.' I don't think that's unreasonable even though we're doing the hosting, so long as our software can actually be run on their own infrastructures, if they want to for test purposes.

DS: Can you give me a concrete example of what a customer ASP might be doing.

BB: Sun, NetBeans, or anything behind Netbeans' source count.

DS: So Sun with NetBean's .org is using Sourcecast to increase the value of NetBeans.

BB: I think this is a great success story because they found ten or fifteen outside developers with CVS commit access. We also have lots of companies now that are using the NetBeans code base as the base for their own commercial IDE, which is really cool and interesting.

DS: So, in the context of your relationship with Sun you're sorting out what's open, what's closed, and working out a business and development model inside the relationships you have with customers, while at the same time looking toward potential customers.

BB: They're NetBeans guys, right? All hard-core Java players. They know the site is written mostly using Java — aside from the CVS server and stuff like that. So when they see a problem, they have the technical ability, they have the code in front of them, to dive in and find out what is wrong. And when you don't give them the source code, they don't have a problem with it.

DS: So you're describing the way relationships actually work in this part of the development bazaar.

BB: Perhaps one way to think of it is, we're focussed on a very few but very valuable customers. We've always had very deep relationships with them. When you push something out like an open source project, you risk involvement with many more people who are not your customers. This can soak up a lot of energy that has nothing to do with running a business, and developing relationships with customers alongside developing code. Worse, you face the prospect of potential customers using open source as a reason not to become customers. I certainly won't call those people idiots, as Eric Raymond did. But he's right that companies are saying they would rather not outsource their work to a wide-open community. They prefer to give most projects to internal engineers. They like the idea of building knowledge in house and outsource the uninteresting stuff.

DS: So what I'm hearing from both you and Eric is that the market is telling us something about the commercial value of community-developed open source code.

BB: Yes. It's also telling us something about how companies can work openly with customers, and about what's best to do just with customers -- and what's best to do with a whole open source development community.

DS: Who else is doing anything similar and what are the differences?

BB: I think most companies that are trying to walk the line between what's proprietary and what isn't. Even Red Hat. Do they give away the source to the server side of the Red Hat network? They shouldn't. I would be hard-pressed to find any company that gives away everything that it writes. There may be examples of small independent consulting companies who have built a business making modifications to a particular open source package. But I would imagine most companies with aims to have any measure of commercial success are going to look at drawing that line carefully. Drawing it is a form of calculus.

DS: You need fuzzy logic.

BB: Yeah, it's a form of measuring risk versus return. Our decision was based on finding the return wasn't matching the risk and opportunities.

Doc Searls is Senior Editor of Linux Journal.